Patient outcome quality indicators for older persons in acute care: original development data using interRAI AC-CGA

Background A range of strategies are available that can improve the outcomes of older persons particularly in relation to basic activities of daily living during and after an acute care (AC) episode. This paper outlines the original development of outcome-oriented quality indicators (QIs) in relation to common geriatric syndromes and function for the care of the frail aged hospitalized in acute general medical wards. Methods Design QIs were developed using evidence from literature, expert opinion, field study data and a formal voting process. A systematic literature review of literature identified existing QIs (there were no outcome QIs) and evidence of interventions that improve older persons’ outcomes in AC. Preliminary indicators were developed by two expert panels following consideration of the evidence. After analysis of the data from field testing (indicator prevalence, variability across sites), panel meetings refined the QIs prior to a formal voting process. Setting Data was collected in nine Australian general medical wards. Participants Patients aged 70 years and over, consented within 24 h of admission to the AC ward. Measurements The interRAI Acute Care – Comprehensive Geriatric Assessment (interRAI AC-CGA) was administered at admission and discharge; a daily risk assessment in hospital; 28-day phone follow-up and chart audit. Results Ten outcome QIs were established which focused on common geriatric syndromes and function for the care of the frail aged hospitalized in acute general medical wards. Conclusion Ten outcome QIs were developed. These QIs can be used to identify areas where specific action will lead to improvements in the quality of care delivered to older persons in hospital. Supplementary Information The online version contains supplementary material available at 10.1186/s12877-024-04980-9.


Background
Older persons who are hospitalized for acute illnesses are at risk of both losing independence and institutionalization [1,2].A range of strategies are available that can improve their outcomes at discharge, particularly in relation to the ability to perform basic activities of daily living and discharge to long term residential care [2].The challenge is in identifying areas where limited time and resources should be focused to make appropriate changes.
Quality assurance activities can be complex and multifactorial.Prior literature tempers expectations between process and outcomes [3].It can be challenging to record a link between changed processes and improved outcomes.Despite this, measuring patient outcomes and improving clinical practice to optimize patient outcomes in the long term are important.Quality indicators (QIs) (structure, process and outcome) are one way to measure quality of care [4].QIs are commonly used in hospitals, but there are no published sets of outcome QIs for geriatric syndromes and function for the care of older persons in acute care (AC) prior to the interRAI AC QIs [5][6][7].
QI development is most effective when it includes expert opinion, a review of the scientific literature and field study data [8,9].Measuring quality of care based on patient outcomes has inherent appeal, but there are major concerns.There are attribution problems (difficulties measuring the aspect of quality precisely or exclusively) [1]; and problems with inaccurate trigger rates (triggering based on other random factors and resulting in inaccurate results).Depending on the location of the hospital, socio-economic and health demographic factors will also impact the bed-use and presentation profile.Any development of outcome QIs must take these factors into account [10] because a QI that measures a hospital's performance is only meaningful when it measures factors that hospital can control, and includes mechanisms to adjust for factors that hospital cannot control.
The aim of this study was to develop outcome oriented QIs in relation to common geriatric syndromes and function for the care of the frail aged hospitalized in acute general medical wards.These QIs have since been adopted into an international assessment system for AC and a supplementary set created for use in a surgical setting.This paper outlines the original development project.

Method
This project was carried out in three phases: Development of a preliminary QI set; field study; analysis and compilation of a definitive QI set.The research methodology for this project has been described in detail elsewhere [10].A truncated description is provided here.

Literature Summary
A scoping review of the scientific literature pertaining to adverse geriatric outcomes in acute hospitals was compiled.Publications with the highest levels of evidence in each domain of interest were identified together with existing guidelines and published QIs relevant to hospitalized older persons (with an interpretation of methodological quality and applicability).

Expert panels
Given the scope of the project, two expert panels were required: panel one focused on generic outcomes for older persons (e.g., continence) [Acute Care Panel, Supplementary Box 1]; and panel two focused on cognitive health (e.g., the recognition of cognitive impairment) [Dementia Care Panel, Supplementary Box 2].Expertise criteria for panel participation was established a priori.Experience included: allied health, consumer advocacy, dementia, general medicine, geriatric medicine, gerontic nursing, quality research methodology, and AC public hospital health care.Participating sites for data collection were recruited from two Australian states (Queensland, Victoria), and coordinators at each site participated in the expert panels.Based on matching site coordinator's experience with the a priori list, any remaining gaps were filled by invitation from the research team.
Expert Panel Meetings.The literature summaries were divided, based on relevance, between the two expert panels.Initially, each panel met for two days to review their respective summaries and conceptualise potential QIs.Panel members recommended QIs according to predefined criteria: being amenable to change, within the control of the hospital, had evidence to support a link between the actions of a hospital and the outcome, and which were clinically significant for an older person and therefore warranted attention.Each QI was defined (i.e.identifying the variables needed for calculating the numerator, denominator, inclusion, and exclusion criteria).The QIs were linked with variables from the interRAI AC -Comprehensive Geriatric Assessment (AC-CGA) tool [11] for data collection in the first instance, to score the QIs prior to evaluation in phase 3.

Phase 2: Field Study
Patients aged 70 years and older admitted to nine Australian AC general medical hospitals, who were likely to stay in hospital for at least 48 h, were invited to participate.The data collection took an average of 7 months (range 4-10 months).Data collection period by hospital type (tertiary or regional) is shown on a radar chart in the Supplementary Fig. 1.Hospitals included five metropolitan tertiary teaching hospitals and four regional hospitals.Ethics approval was obtained from the relevant organisations.

Data collection
Each hospital nominated one or two registered nurses with gerontic experience to work full-time on the project as data collectors during the study period.One fulltime registered nurse with gerontic experience, trained in project data collection, was rostered each day per site for 7.5 h of data collection a day across five days each week (Monday to Friday) during the data collection period.All nurse assessors attended a three-day training program at the Centre for Health Services Research (CHSR) prior to commencing data collection.Training involved understanding the project protocol and minimising potential research bias, administration and scoring of the formal assessment tools, and the completion of the study recruitment database.A regular teleconference schedule was established for interaction between sites during the data collection period and protocols for data storage and transfer were put in place.All site research nurses were trained to administer the interRAI AC-CGA by the same experienced interRAI assessment trainer.Previous studies have shown very high levels of reliability on observed agreement for the interRAI AC-CGA [12,13].
Demographic data was collected on patients who consented to participate in the study.The interRAI AC-CGA was administered within 24 h of admission to the ward for consenting patients, with daily follow-up and review at discharge.At 28 days, a follow-up telephone call to the patient's or carer's home was made to collect data on readmission and post-discharge change of living arrangement, this included the Mini-Mental State Examination 22-item telephone version (ALFI-MMSE 22-items) [14,15].The data collected at 28 days was part of demographic data and was not used in the calculation of any QIs.Verification of re-admission, including diagnostic codes, was obtained by State data custodians or the relevant hospital data custodian.

Phase 3: Analysis and final consultation (with expert panels) Statistical analysis
Data was entered into IBM SPSS Statistics v26 2019.General descriptive statistics (frequencies, standard deviations and means) were used to describe characteristics of enrolled participants and to compare between hospitals, using demographic data, and cognitive and functional scales derived from the interRAI AC-CGA.
Each QI was scored for the total sample, and for each individual hospital.A trigger rate (percentage) was calculated for each QI (the proportion of patients who triggered the indicator event (the numerator) divided by the total number of eligible patients (the denominator)).
The expert-panel-identified covariates were analysed for statistical relevance to the sample by presenting the QI trigger rates when the covariate was present and absent.This was followed by descriptive statistics (odds ratios with 95% confidence intervals, and p-values) to identify statistically significant factors for each QI.All covariate information was provided, noting that multiple risk adjustments cannot be applied simultaneously to this sample given the small sample size.
Using the RAND-UCLA consensus method [16] two rounds of voting for each panel were completed.Analysis of voting round data was completed after each round.For each QI the following were calculated using the panel members' individual votes of 1-9: the 30th and 70th percentile, the Interpercentile Range Adjusted for Symmetry, the Interpercentile Range, Interpercentile Range Central Point, the Asymmetry Index, median and mean deviation from the median.This data were used to apply decision rules, defined a priori, to identify valid or invalid QIs [17].Only votes from round 2 were utilised to determine the final QI set.

Expert Panel
A report was prepared for each panel (there was no overlap of indicators between panels) with a description of the preliminary indicators suggested from Phases 1 and 2, feedback provided by the interRAI AC Network and the interRAI Instrument Committee, and any recommendations (Supplementary Fig. 2).Any new research published in the interim was also provided.Each panel held a second two-day workshop to scrutinise the preliminary QIs which had now been tested in the field.QIs were modified based on recommendations from the expert panel following interpretation of the field data, evidence from the literature, and expert opinion.
Risk Adjustment: Risk adjustment was considered for each QI separately at the final expert panel meetings.Risk adjustment was approved if there was agreement amongst the panel members that factors, outside the control of the hospital, would impact the QI's outcome, and those factors couldn't be dealt with by excluding patients from the denominator.Risk adjustment was not approved in situations where it was recognised that certain patient characteristics did pose extra challenges but this didn't obviate the extra care that was required to ensure patients' safety and well-being.For example, the Dementia Care panel recognized that patients with dementia were at higher risk of delirium in hospital, but this did not justify risk adjustment for the delirium QI.It was important that staff identify these patients and provide care to minimise the risk of delirium rather than accept that it was inevitable.For each QI, the prevalence of the outcome across all patients in hospital datasets, after exclusions, is the facility-level observed QI score.Risk adjustment was calculated, when the final set of outcome QIs was identified, using logistic regression models [10,18].

Voting
Each panel voted on an independent set of QIs which were combined to establish the final outcome QI set.Each member voted on a scale of 1-9 with reference to specific criteria.A second voting sheet was prepared individually for each panel member which showed the spread of panel votes for each QI and the personal vote of that panel member, along with the QI data analysis (Valid or not, etc.).A teleconference was held to discuss the round 1 results.Where the votes for a QI was wide-spread, it generally indicated a misunderstanding, a poorly constructed QI, or diverse opinion amongst the panel members.The teleconference enabled clarification and rectified any misunderstandings.Each panel member voted a second time to conclude the final outcome AC set.

Results
Data from 643 patients at nine hospitals were analysed and used to calculate each individual QI.No patients withdrew during the in-hospital data collection.Characteristics of the included sample (Table 1) is provided.

The Research Collaboration for Quality Care (RCQC) Outcome QIs for Older Persons in AC
Ten outcome indicators were developed for the AC QIs for Older Persons.Table 2 describes each QI.Each QI has been classified according to the Institute of Medicine (IOM) framework, which is a useful way of considering the domains of care, and how this set of outcome QIs sit within those domains [19,20].The domains of care, as defined in the IOM framework are: safety, effectiveness, patient-centeredness, timeliness, efficiency, and equity.Figure 1 presents the QIs and displays the observed between site variation of scores within each QI.Table 3 provides raw summary data for each QI.In all cases, the QIs are interpreted by understanding that a lower score reflects a better outcome for the patient and the hospital.

RCQC Outcome QIs for older persons in AC:
1.The proportion of female patients with a new urinary catheter on admission.Functional decline guidelines recommend careful assessment of the need for an indwelling urinary catheter prior to use given the evidence of a causal link between indwelling catheter use and urinary tract infections [21,22].The focus of this QI is on female patients as it was felt that clinically there was less justification for using catheters in women than men and that this better represented quality of care (face validity) than a QI for all patients.2. The proportion of patients with delirium-indicating behaviors present at discharge.There is evidence for delirium reducing interventions in AC.Australian functional decline guidelines (expert opinion levels of evidence) recommend consideration of transitional care needs and community-based strategies for people discharged from hospital and residential care to take into account delirium issues [21,22].This QI considers the potential risk of discharging older people from hospital if they have delirium, without firstly being aware that delirium is present, and second using clinical judgement to decide if discharge is the best course of action.The panel recognises that in some cases, with careful discharge planning, the best choice for a patient with delirium may be discharge.

The proportion of patients discharged with
worse levels of cognitive function compared with premorbid levels.Declining Mini-Mental State Examination scores have been shown to predict mortality [23].Interventions can change the outcome and assist in maintaining cognitive functions [24].There is variation in recoverable cognitive dysfunction amongst older patients in AC [25].This QI was included by the panel because of the impact of cognitive decline, over and above that of delirium, for older patients.the outcome pressure ulcer incidence [30].There is strong evidence to link specific practices in to reducing the risk of pressure ulcers (in patients with high risk) [21,22].The panel recognized that the incidence of pressure ulcers was small in this study but that the clinical impact was significant and therefore it was included as an indicator of quality of care.8.The proportion of patients who fell (at least once) during the hospital episode.Some falls are preventable while some are not preventable [31].There is sound evidence to support the use of multifaceted approaches to reduce the falls rate in hospital settings.There is some evidence to support the use of specialist geriatric wards to reduce falls incidence and fall-related injuries among postoperative hip fracture patients [32,33].The panel recognized that the incidence of falls was small in this study but that the clinical impact for older persons was significant and therefore it was included as an indicator of quality of care.9.The proportion of patients with prolonged length of stay.Generally, injured older adults (> = 65 years) have an average length of hospital stay, including admissions to ICU, exceeding 10 days.A retrospective study of the characteristics and outcomes of injured older adults after hospital admission (N = 6,069) from the Queensland Trauma Registry identified that the average length of stay (including patients in ICU) was eight days (interquartile range 5-15), with 33.8% of cases with a major injury developing a complication [34].
There is evidence to indicate that the risk of iatrogenic complications is higher if an older person is hospitalized for longer than one week.These complications are frequently preventable and are often associated with a rapid decline in muscle strength, aerobic capacity and pulmonary ventilation that older patients experience [35].For the majority of older patients, 'waiting for placement' is not a contributor to their length of stay.Most older people are discharged to their usual place of residence [36].10.The proportion of community-dwelling patients discharged to long-term care.There is good evidence to support the use of multidisciplinary interventions that include a component of exercise to reduce in-hospital length of stay, increase the proportion of patients discharge directly home and reduce costs of stay for older acute hospitalised inpatients [21,22].A review focused on patients regaining functional independence in the acute setting following hip fracture concluded that patients who stayed longer in the acute setting, received physical therapy more than once a day had improved odds of regaining independence in bed mobility, transfers and ambulation.Patients who regained independence along with use of physical therapy services had improved odds of being discharged directly to the home from the acute setting [37].
No covariates for risk adjustment were recommended for three QIs: Delirium, Self Care, and Pain.The remaining QIs each had covariates recommended for risk adjustment (Table 4).The initial step for industry uptake has been swift.The final set of QIs were presented to the interRAI Instrument Committee.The RCQC Patient Outcome QIs for Older Persons in AC were formally endorsed by interRAI and adopted (except for the cognition indicator) for use with the interRAI AC suite [7,11] with modification recommended for the surgical context [38].Software vendors who provide electronic versions of the interRAI AC now also include algorithms to run the AC outcome indictors derived from the data collected by staff when completing care assessments.

Discussion
The aim of this project was to develop outcome-oriented QIs in relation to common geriatric syndromes and function for the care of the frail aged hospitalized in acute general medical wards derived from the interRAI AC-CGA assessment tool clinical data items.The interRAI assessments were selected because they are implemented as care focused sources of data collection.From that care assessment data, a sub-set of items are used to score the QIs, but it is assumed that the assessment is collected in full initially by care staff for the primary purpose of delivering care to a patient in a health service.There is no additional burden of collecting data for administrative or quality care purposes.Many of the QIs can be scored without the interRAI AC-CGA, or they can be scored using a measure the facility currently uses if only internal benchmarking is occurring.QIs have been clearly marked in Tables 2 and Supplementary Table 1 to indicate where this applies.
Ten outcome QIs have been developed, with nine formally endorsed by interRAI for use in the interRAI AC suite (except for the cognition indicator as it was decided delirium was the primary quality issue relevant in AC at this time).The interRAI AC suite is a set of tools, primarily used by nurses at a ward level for risk assessment, to guide clinical care planning and preparation for specialist consultation.The QIs were developed to support improvements in clinical care at the ward level.They can be used to benchmark existing levels of care, and measure change after the implementation of interventions.Future work in quality improvement and outcome QIs will focus on understanding the extent to which staff self-direct the modification of their clinical activities when they are of the QI scoring.When the study was carried out originally the primary data collection tool was the interRAI AC-CGA (a longer assessment).There is now a shorter nursing assessment, a 15-min interRAI AC, and supplementary surgical QIs have been developed.The Supplementary Table 1 illustrates how the QIs can be scored using administrative or clinical data, or interRAI data (i.e.Some QIs can be scored without the interRAI AC as they are straightforward prevalence or incidence indicators).These are listed in the Supplementary Table 1 and indicated with an * including bladder catheter, pain, skin integrity, falls, prolonged stay, institutional placement.At present, some QIs are scored using algorithms calculated from assessment data in the interRAI AC-CGA and the interRAI AC (delirium, cognition, self-care, mobility).Detailed information regarding the scoring is available in the interRAI clinical and applications manual for the interRAI AC [39] which is sourced at the interRAI website (www.inter RAI.com).For example, QIs scored using scales inherent in the interRAI AC, and if the QI was to be implemented and scored using a different system (such as a cognitive tool, rather than the interRAI cognitive CPS scale), the benchmarks in this paper would not be comparable.The QIs scored using interRAI scales or specific items are marked as such in Table 2 and the Supplementary Table 1: these are the Delirium, Cognition, Mobility and Self Care QIs.For organisations not using the interRAI assessment to score these QIs, it is possible to use validated assessments as measures for internal purposes (for example, when scoring the delirium QI) but this will not enable immediate comparison for external benchmarking purposes as different measures have been used, nor can the data in this paper be used as a baseline.
We acknowledge the limitation of this project, that it was carried out in general medical wards at nine Australian hospitals in two States.We recognize that geriatric specific wards would have the potential to score differently as it is possible that the specific training and expertise that staff have in these units would change the outcomes for older persons (possibly improved outcomes; lower scores).The application of these QIs in general medical wards internationally may also generate different results, despite this, generalizability is not impacted as the criteria for quality of care is the same around the world.We were also limited in the information we could collect regarding participating patients and did not have consistency across all sites.This information would have increased our understanding of the generalisability of the findings and will be a strong focus for future work.Second, this paper describes the development of a set of outcome QIs.Further research to show the extent to which these indicators respond to interventions designed to improve patient outcomes is needed to validate their use in benchmarking quality improvement activities.The current set of indicators have been selected based on face validity in relation to responsiveness (i.e.there is published evidence that interventions can change the outcome in the area of interest).The interRAI AC-CGA has been used as the minimum data set for the development of the QIs (and they can also be scored on the companion interRAI AC).A strength in this approach is that interRAI assessments are an accepted international standard for minimum datasets and in many countries the assessments are mandated as part of health care data collection in a variety of health care settings.This creates an opportunity for these QIs to be utilised for international benchmarking in acute care in the future.Conversely, uptake maybe hindered by organisations which choose not to access licensed standardised assessment tools for their data collection and patient assessment.For those organisations, sufficient information is in the public domain for internal benchmarking activities utilising these QIs without the interRAI AC assessment but this limits the potential for international benchmarking.
Prior to this study, there were no published outcome indicators for older persons in AC.This work was a first step towards supporting improvement in the care of older persons in an acute setting.In their current format, these QIs can be applied in an acute hospital ward and the results can be compared with the field study data listed here as a benchmark.Poor scores on particular QIs can help to focus quality assurance time and resources to improving care where it is needed most.

Conclusions
Two expert panels were formed to review the scientific literature and interpret field study data to develop a set of outcome QIs.Ten outcome QIs were developed.These QIs can be used to identify areas where specific action will lead to improvements in the quality of care delivered to older persons in hospital.

The proportion of patients discharged with worse mobility levels compared with pre-mor
4.

proportion of patients with pre-hospital decline who failed to return to preadmission function (or better) by discharge
[27]ere is a significant body of evidence that shows older people decline in function by the time of discharge from AC, and that this negatively impacts quality of life of hospitalized older people[27].Recommendations of the Australian Functional Decline

proportion of patients with no pre-morbid pain who reported both pain at admission and unimproved pain at discharge
. There is evidence that improved strategies to manage pain (i.e.qual-

Table 1
Characteristics of patients in the studyData are presented as median (IQR) for continuous measures, and n (%) for categorical measures.a Increased risk of activities of daily life (ADL) decline -ADL-SF score from premorbid between 2 and 16 or CPS score at admission between 2 and 5. Adverse outcome denotes the event of one or more of either fall, pressure area, decline in activities of daily living (assessed daily), delirium onset b

Table 2
[20]aboration for Quality Care -patient outcome quality indicators for older persons in acute care: definitions[20] [28,29]rovement collaborative program) improve processes of care (such as improved use of pain assessment tools), but only limited evidence of links to patient outcomes (i.e.sufficient pain treatment for patients)[28,29].7.The proportion of patients with a new or wors-ening pressure ulcer at discharge when compared with admission.A review of studies reporting the implementation of pressure ulcer reduction strategies (n = 16) reported a positive reduction in N Nominator, D Denominator, C Covariate, E Exclusion, IOM Institute of Medicine framework a Derived using an interRAI Scale or specific variable; †Not in the interRAI Acute Care Quality Indicator Set

Table 2
(continued)Field study data of outcome indicators for older adults, ordered by lowest to highest performing in terms of overall trigger rate.Note: The size of the marker is scaled according to the denominator

Table 3
Raw summary data each quality indicator, including trigger rates (with p-values) for the whole sample, and each hospital

Table 4
Recommended covariates and examination of factors associated with risk adjustmentCare assessment and planning targeting patients at all times have no covariate recommendations: Delirium, Self-Care, and Pain.QI quality indicator *Statistically significant (p < 0.05)